Papers with majority voting
Copied to clipboard
| Challenge: | Existing tools do not surface subtler psychosocial harms, nor provide explainable rationales that practitioners need. |
| Approach: | They propose an open-source system that lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
| Outcome: | The open-source DialogGuard system lets practitioners inspect, stress-test, and create audit trails for prompted LLM agents across five psychosocial safety dimensions. |
Copied to clipboard
| Challenge: | Current “sample and select” methods rely on majority voting to score answers . however, when tasks have many distinct and valid answers, selection by voting requires a large number of samples. |
| Approach: | They introduce a method that replaces SC's discontinuous scoring with a continuous score computed from model likelihoods to increase selection even when actions are sparsely distributed. |
| Outcome: | The proposed method improves performance and efficiency on long-horizon interactive tasks by replacing SC’s discontinuous scoring with a continuous score computed from model likelihoods. |
Copied to clipboard
| Challenge: | Existing studies on social media bias detection focus on fine-tuning models specific to particular datasets and testing them on corresponding test sets. |
| Approach: | They propose a general bias detection framework, IndiVec, built upon large language models and vector databases. |
| Outcome: | The proposed framework outperforms baseline methods on four political bias datasets and provides explicit top-k indicators to interpret bias predictions. |
Copied to clipboard
| Challenge: | Existing models for named entity recognition (NER) are based on large-scale labeled datasets, which always obtain using crowdsourcing. |
| Approach: | They propose a CONfidence-based partial Label Learning method to integrate prior and posterior confidences for crowd-annotated named entity recognition models. |
| Outcome: | The proposed model improves on real-world and synthetic datasets compared with baselines. |
Copied to clipboard
| Challenge: | Visual communication relies heavily on images and short texts to grab a viewer's attention and convey a message in the most efficient way. |
| Approach: | They propose a model that employs end-to-end label distribution learning on crowd-sourced data and predicts a selection distribution, capturing the inter-subjectivity and ambiguity of the input. |
| Outcome: | The proposed model captures the inter-subjectivity and ambiguity of the input and can be transformed to single-label learning by mapping labels to absolute labels via majority voting. |
Copied to clipboard
| Challenge: | Recent studies indicate that learning from individual annotations outperforms learning from aggregated labels, though they require a considerable amount of annotation. |
| Approach: | They propose to use a multi-head model to learn from disagreements in an active learning setting to identify annotators with a single head. |
| Outcome: | The proposed model outperforms a single-head model in terms of uncertainty estimation and prediction while saving 70% of the annotation budget. |
Copied to clipboard
| Challenge: | Disagreement can reflect different aspects of linguistic annotation, from annotators’ subjectivity to sloppiness or lack of context to interpret a text. |
| Approach: | They propose a taxonomy of possible reasons leading to annotators' disagreement in subjective tasks and manually label part of a Twitter dataset for offensive language detection in english following this taxonomies. |
| Outcome: | The proposed taxonomy of disagreements in linguistic datasets can be used to assess how accurate tweets belonging to different disagreement categories can be classified as offensive or not. |
Copied to clipboard
| Challenge: | Corpus-based studies on acceptability judgements have always been popular thanks to the release of the CoLA corpus, a large-scale corpus of sentences extracted from linguistic handbooks as examples of acceptable/non acceptable phenomena in English. |
| Approach: | They present a 3D video game that was used to collect acceptability judgments on italian sentences and compare them with experts’ acceptability judgements. |
| Outcome: | The proposed game compares the annotations of Italian sentences with those of experts and shows that they are more reliable than crowd-sourced annotations. |
Copied to clipboard
| Challenge: | Large language models with search capabilities often exhibit miscalibrated confidence, causing incorrect answers with high certainty. |
| Approach: | They propose a reasoning-primary framework that integrates search operations into chain-of-thought generation while maintaining explicit confidence calibration. |
| Outcome: | The proposed framework improves accuracy and reliability of large language models with search capabilities. |
Copied to clipboard
| Challenge: | Successful Named Entity Recognition models fail on texts from some special domains, for example, Chinese addresses and e-commerce titles. |
| Approach: | They propose to enhance NER models with correlated samples to help the text understanding . they draw correlated texts by the sparse BM25 retriever from large-scale in-domain unlabeled data . |
| Outcome: | Empirical results show that NER models can be enhanced with correlated samples . the proposed model can be used to reason out the correct answer on hard cases . |
Copied to clipboard
| Challenge: | Recent studies suggest that self-reflective prompting can significantly enhance the reasoning capabilities of Large Language Models (LLMs). |
| Approach: | They propose guidelines for when to implement self-reflection in Large Language Models. |
| Outcome: | The proposed approach improves the reasoning capabilities of Large Language Models under a more stringent evaluation setting, and reduces tendency toward majority voting. |
Copied to clipboard
| Challenge: | Existing methods rely on majority voting or criteria expansion to capture detailed and detailed details, often leading to incomplete outcomes. |
| Approach: | They propose a method which introduces additional crowd responses to compare with the candidate responses, thereby exposing deeper and more comprehensive details within the candidate answers. |
| Outcome: | Experiments show that the proposed method improves evaluation reliability and achieves an average gain of 6.7% across five benchmarks. |
Copied to clipboard
| Challenge: | Existing text decoding methods struggle to produce high-quality text . Greedy and beam search suffer from text degeneration and linguistic diversity issues . |
| Approach: | They propose a family of decoding methods based on minimum bayes risk minimization to address diversity-quality trade-offs in open-ended natural-language generation. |
| Outcome: | The proposed methods improve diversity-quality trade-offs on open-ended natural-language generation tasks. |
Copied to clipboard
| Challenge: | Recent work on Chain-of-Thought reasoning requires manual prompts to guide the model. |
| Approach: | They propose a general decoding strategy that generates CoT-style reasoning paths without prompts. |
| Outcome: | The proposed method maintains strong performance on fixed and free QA tasks and achieves significant improvements on free qa. |
Copied to clipboard
| Challenge: | Existing approaches to self-improvement rely on external supervision signals in the form of seed data and/or assistance from third-party models. |
| Approach: | They propose a framework for generating high-quality synthetic question-answer data in a fully autonomous manner. |
| Outcome: | The proposed framework generates high-quality synthetic question-answer data in a fully autonomous manner. |
Copied to clipboard
| Challenge: | Existing methods for annotating data are limited by ambiguity and lack of context in data samples. |
| Approach: | They challenge the traditional approach of annotating data by only providing a single label for each sample and annotator disagreement is discarded . instead, they use additional annotation information such as confidence, secondary label and disagreement to generate soft labels. |
| Outcome: | The proposed method improves model performance and calibration on the hard label test set. |
Copied to clipboard
| Challenge: | In a cross-dataset evaluation, models trained on BanNERD consistently outperformed those trained on four existing Bangla NER datasets. |
| Approach: | They propose to use Bangla as a language to create the most extensive human-annotated and validated Bangla NLP dataset. |
| Outcome: | The proposed method outperforms existing methods on Bangla NER datasets and performs competitively on English datasets. |
Copied to clipboard
| Challenge: | sarcasm detection datasets focus on intended, rather than perceived sarcasm, but there is no comparison between human and machine performance. |
| Approach: | They collect author-annotated sarcasm datasets that focus on intended, rather than perceived sarcasticism . they compare human-level benchmarks to that of state-of-the-art sarkasmatic detection systems . |
| Outcome: | The proposed datasets compare human and machine performance on sarcastic tasks in English and Arabic. |
Copied to clipboard
| Challenge: | Existing approaches to enhance the reliability of Large Language Models (LLMs) in complex reasoning tasks are limited by their limitations. |
| Approach: | They propose an uncertainty-aware framework to enhance the reliability of Large Language Models . they quantify the confidence of intermediate answers and evaluate the reliability based on these confidences a way that reflects the reliability. |
| Outcome: | The proposed approach improves accuracy of large language models in math and open-domain tasks by 7.4% and 5.8% over baseline approaches. |
Copied to clipboard
| Challenge: | Extensive experiments on six standard mathematical data sets and three LLMs show that FOBAR achieves state-of-the-art performance. |
| Approach: | They propose to combine forward and backward reasoning to verify candidate answers . they propose to use a template to mask a number and ask the LLM to answer a backward question . |
| Outcome: | Experiments on mathematical data show that proposed backward reasoning outperforms Self-Consistency. |
Copied to clipboard
| Challenge: | Existing approaches to label aggregation fail to capture subjective annotations and can lead to biases. |
| Approach: | They propose annotator-aware representations for text for subjective classification tasks that involve learning representations of annotators. |
| Outcome: | The proposed model improves on metrics that assess the performance on capturing individual annotators’ perspectives. |
Copied to clipboard
| Challenge: | Existing approaches to improve the generalization of large language models are using Supervised Fine-Tuning (SFT) this approach does not show sufficient generalization ability because it only relies on the given CoT data. |
| Approach: | They propose to use Chain-of-Thought annotations to train Large Language Models using supervised fine-tuning to improve generalization. |
| Outcome: | The proposed approach outperforms SFT on GSM8K, MathQA, and SVAMP datasets and shows a superior generalization ability. |
Copied to clipboard
| Challenge: | Existing knowledge graph methods lack adaptability in knowledge-intensive tasks with multiple entities and implicit multi-hop relations. |
| Approach: | They propose a zero-shot framework to enhance LLM reasoning in multi-entity relation tasks. |
| Outcome: | DS-MHP outperforms baselines and state-of-the-art methods on 12 datasets spanning commonsense, logical, symbolic, and arithmetic reasoning. |
Copied to clipboard
| Challenge: | Increasing the number of agents improves performance, while more discussion rounds before voting reduces it. |
| Approach: | They propose two new methods to improve multi-agent debates by increasing agent diversity and reducing discussion rounds before voting. |
| Outcome: | The proposed methods improve task performance by up to 3.3% with AAD and up to 7.4% with CI. |
Copied to clipboard
| Challenge: | Existing approaches to selecting a single high-quality output from multiple generations are limited in their applicability and efficiency. |
| Approach: | They propose a method that generalizes majority voting to open-ended text generation . modeX leverages structural information across multiple generation paths to select a "modal" output . |
| Outcome: | The proposed framework outperforms standard single- and multi-path baselines in open-ended tasks. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly adopted across real-world applications . traditional evaluations rely on expensive, domain-specific ground-truth labels . obtaining labeled data is expensive, time-consuming, and often requires domain expertise . |
| Approach: | They propose a ground-truth-free evaluation framework focused on reasoning consistency and instruction following. |
| Outcome: | The proposed framework outperforms existing label-free methods, including majority voting, triplet ranking, and peer-review approaches. |
Copied to clipboard
| Challenge: | a recent study shows that crowdsourcing is becoming mainstream to create bilingual dictionaries . the number of people who can speak multiple low-resource languages is limited and the average ability of workers is low. |
| Approach: | They propose a method to aggregate the answers of evaluation tasks by majority voting . they use hyper questions to evaluate the reliability of workers and task-allocation method to select high-quality workers . |
| Outcome: | The proposed method improves quality of bilingual dictionaries by integrating answers by majority voting. |
Copied to clipboard
| Challenge: | Existing methods for BN structure learning are limited by the size of the BN. |
| Approach: | They propose a method for Bayesian Networks (BNs) structure elicitation that initializes several LLMs with different experiences and queries them to create a structure. |
| Outcome: | The proposed method performs better than the existing method with one of the three studied LLMs, but the performance decreases with the increase in BN size. |
Copied to clipboard
| Challenge: | Existing verifiers operate on the surface text or on confidence proxies derived from token probabilities, which can be brittle. |
| Approach: | They propose a training-free, non-parametric verifier that summarizes each reasoning trace by an activation delta and compares it to two class centroids computed from labeled experience. |
| Outcome: | The proposed model improves selection and reranking on large and less-calibrated models. |
Copied to clipboard
| Challenge: | Existing methods to train language models rely on manual design, perplexity, or careful prompt engineering. |
| Approach: | They propose a method that automatically mines criteria from human preferences for data quality with only 30 human-annotated pairs and performs efficient data selection. |
| Outcome: | The proposed method improves on human-annotated test sets and shows high accuracy on code, math, and logic domains. |
Copied to clipboard
| Challenge: | Large language models generate reasoning paths before final answers, but learning such a path requires costly human supervision. |
| Approach: | They propose a method that fine-tunes LLMs to prefer reasoning paths with high confidence . they propose 'cORE-PO' that fine tunes Lms to choose high-quality reasoning paths . |
| Outcome: | The proposed method improves the accuracy of outputs on four in-distribution and two out-of-difference benchmarks. |
Copied to clipboard
| Challenge: | Previously, studies on stance and dogmatism in user conversations have focused on training models using annotated datasets at the post level, treating each post as independent and randomly sampling posts from conversation threads. |
| Approach: | They build a dataset for studying user opinion fluctuations in 764 long multi-user Reddit conversation threads, called USDC. |
| Outcome: | The proposed dataset analyzes user opinion fluctuations in 764 long multi-user Reddit conversation threads. |
Copied to clipboard
| Challenge: | Existing approaches to test-time scaling are limited due to the quality of candidate responses. |
| Approach: | They propose a new metric to quantify the relative improvement of self-refinement beyond majority voting. |
| Outcome: | The proposed method achieves state-of-the-art performance across five benchmarks over other methods. |
Copied to clipboard
| Challenge: | Translating natural language questions into SQL is a core challenge in natural language understanding and human-computer interaction. |
| Approach: | They propose a reinforcement learning framework and model family to generate accurate, executable SQL using a lightweight reward signal based solely on execution correctness. |
| Outcome: | The proposed framework outperforms previous versions of 70B-class systems and achieves state-of-the-art execution accuracy across six diverse Text2SQL benchmarks. |
Copied to clipboard
| Challenge: | Recent studies have shown that scaling test-time compute can also effectively improve reasoning. |
| Approach: | They propose a probabilistic method to efficiently predict scaling performance and identify the best prompting strategy under large sampling times. |
| Outcome: | The proposed method significantly improves the scaling performance of majority voting on large language models. |
Copied to clipboard
| Challenge: | Existing approaches to selecting reliable responses from multiple LLMs often depend on external verifiers, human evaluators, or self-consistency techniques. |
| Approach: | They propose a calibrated log-likelihood-based selection framework to improve multi-LLM performance. |
| Outcome: | The proposed method outperforms majority voting and exceeds self-consistency performance when using a large number of model calls. |
Copied to clipboard
| Challenge: | Existing methods for testing time scales treat reasoning traces or tokens equally, ignoring substantial variations in trajectory quality and localized logical failures. |
| Approach: | They propose a chronological reasoning scorer that models each trajectory as a time series. |
| Outcome: | The proposed method achieves relative improvements of 34.21% over Pass@128 and 22.70% over Maj@135 on HMMT25, highlighting its effectiveness. |
Copied to clipboard
| Challenge: | Existing work has focused on the (un)intended leakage of sensitive information through LLM outputs. |
| Approach: | They propose a threat model that embeds context information into natural-looking outputs via linguistic steganography without requiring explicit control over inference inputs. |
| Outcome: | The proposed model transmits 32-bit secrets with 87% accuracy on held-out prompts and can reach over 97% accuracy using majority voting across three generations. |
Copied to clipboard
| Challenge: | Reliable multi-image geological reasoning is essential for automating expert tasks in remote-sensing mineral exploration. |
| Approach: | They propose a framework that orchestrates planning, execution, and verification agents to decompose, ground, and iteratively refine reasoning steps over geological and hyperspectral image sets. |
| Outcome: | The proposed framework decomposes, ground, and iteratively refines reasoning steps over geological and hyperspectral image sets. |
Copied to clipboard
| Challenge: | Existing evaluations of large language models rely on dataset-based generation accuracy . however, generative correctness does not guarantee discriminative capability to verify solutions . |
| Approach: | They propose a diagnostic framework that explicitly controls context and isolates discriminative behaviors. |
| Outcome: | The proposed framework explicitly controls context and isolates discriminative behaviors. |
Copied to clipboard
| Challenge: | Existing multi-agent debate methods rely on multiple rounds of interaction among agents to reach consensus, and the final output is decided by majority voting in the last round. |
| Approach: | They propose a multi-agent debate framework that eliminates the need for consensus among agents and reconstructs the debate phase by introducing anti-conformity. |
| Outcome: | Experiments on eight benchmark datasets show that Free-MAD significantly improves reasoning performance while requiring only a single-round debate and thus reducing token costs. |
Copied to clipboard
| Challenge: | Large Language Models (LLMs)-based TableQA models exhibit unfaithful behavior where correct answers are derived through erroneous reasoning paths. |
| Approach: | They propose a neuro-symbolic framework to audit LLM reasoning processes . it enforces factual grounding and ensures logical soundness by verifying reasoning chains . |
| Outcome: | The proposed framework outperforms LLM judges in majority voting and rejection sampling with process supervision. |
Copied to clipboard
| Challenge: | RLVR is a paradigm for improving reasoning ability of large language models . but voting results often induce confirmation bias and suffer from sparse rewards . |
| Approach: | They propose a framework integrating model confidence and dynamic subgroup partitioning to address these issues. |
| Outcome: | The proposed framework outperforms recent baselines on multiple models and benchmarks. |
Copied to clipboard
| Challenge: | Recent studies show that Test-Time Scaling (TTS) can improve reasoning performance without retraining the model. |
| Approach: | They propose a step-wise pruning strategy that identifies and removes redundant chains using inter-chain similarity at the thought level. |
| Outcome: | The proposed method reduces inference latency and KVC usage by up to 45% and 26% with R1-Distill while maintaining or improving accuracy. |
Copied to clipboard
| Challenge: | Existing studies in Bangla focus on hate classification while overlooking interpretability. |
| Approach: | They propose to create a dataset with human-annotated labels for banla that contains 19,203 YouTube comments spanning April 2024–June 2025. |
| Outcome: | The proposed dataset outperforms existing datasets on open and closed-source LLMs on interpretability and better understanding of hate speech in linguistically rich yet under-resourced languages. |